Search for: All records

This survey explores the transformative impact of foundation models (FMs) in artificial intelligence, focusing on their integration with federated learning (FL) in biomedical research. Foundation models such as ChatGPT, LLaMa, and CLIP, which are trained on vast datasets through methods including unsupervised pretraining, self-supervised learning, instructed fine-tuning, and reinforcement learning from human feedback, represent significant advancements in machine learning. These models, with their ability to generate coherent text and realistic images, are crucial for biomedical applications that require processing diverse data forms such as clinical reports, diagnostic images, and multimodal patient interactions. The incorporation of FL with these sophisticated models presents a promising strategy to harness their analytical power while safeguarding the privacy of sensitive medical data. This approach not only enhances the capabilities of FMs in medical diagnostics and personalized treatment but also addresses critical concerns about data privacy and security in healthcare. This survey reviews the current applications of FMs in federated settings, underscores the challenges, and identifies future research directions including scaling FMs, managing data diversity, and enhancing communication efficiency within FL frameworks. The objective is to encourage further research into the combined potential of FMs and FL, laying the groundwork for healthcare innovations. 

We study the implicit bias of flatness / low (loss) curvature and its effects on generalization in two-layer overparameterized ReLU networks with multivariate inputs---a problem well motivated by the minima stability and edge-of-stability phenomena in gradient-descent training. Existing work either requires interpolation or focuses only on univariate inputs. This paper presents new and somewhat surprising theoretical results for multivariate inputs. On two natural settings (1) generalization gap for flat solutions, and (2) mean-squared error (MSE) in nonparametric function estimation by stable minima, we prove upper and lower bounds, which establish that while flatness does imply generalization, the resulting rates of convergence necessarily deteriorate exponentially as the input dimension grows. This gives an exponential separation between the flat solutions compared to low-norm solutions (i.e., weight decay), which are known not to suffer from the curse of dimensionality. In particular, our minimax lower bound construction, based on a novel packing argument with boundary-localized ReLU neurons, reveals how flat solutions can exploit a kind of "neural shattering" where neurons rarely activate, but with high weight magnitudes. This leads to poor performance in high dimensions. We corroborate these theoretical findings with extensive numerical simulations. To the best of our knowledge, our analysis provides the first systematic explanation for why flat minima may fail to generalize in high dimensions. 

Abstract Most STEM classrooms overlook the intrinsic conceptual structure of domain content, strategies for improving students’ conceptual structure have promise for improving STEM learning outcomes. This experimental investigation continues the development of the web-based toolGraphical Interface of Knowledge Structure (GIKS)that provides immediate formative feedback as a network of concepts in the student’s essays alongside an expert referent network for comparison and reflection. What should this feedback network look like, especially, should it be more inclusive or small and focused? And is preexisting domain knowledge important for type of network feedback effectiveness? Undergraduate students in a second year Architecture Engineering course, after completing a 2-weeks long lesson on Building with Wood, were randomly assigned to a summary writing task with either Full feedback (a network with 14 central and 12 peripheral terms) or Focused feedback (a network with only the 14 central terms), and then immediately completed a knowledge structure survey. Two weeks later, they completed an End-of-Unit posttest that consisted of a Central-items and a Peripheral-items subtests. A significant interaction of feedback and domain knowledge was observed for post knowledge structure, the low domain knowledge students in the Focus feedback group had the most central link-agreement with the expert and the least peripheral links agreement. On the End-of-Unit declarative knowledge posttest, there was no difference for the Full or Focused feedback interventions, but the high domain knowledge students in both interventions performed significantly better than the low domain knowledge students on the central-items subtest butnoton the peripheral-items subtest. This investigation shows the need for further research on the role of domain-normative central concepts and pragmatically contributes to the design of essay prompts for STEM classroom use. 

Abstract Nanostructured epoxy composite resins have broad usage in adhesives, coatings, composites, and 3D printing. With these materials, careful control of the rheological properties is critical to ensuring that the properties meet their required performance targets. However, it can be difficult to accurately measure the rheological properties. In this work, we establish a method to develop a reliable pre-shear (PS) procedure to repeatably measure the apparent yield stress of the resins, which is critical to ensure the accurate understanding of the material behavior. The resins in this study consisted of an epoxy resin with nanoclay as a shear thinning agent, ionic liquid (1-ethyl-3-methylimidazolium dicyanamide) as a latent curing agent, and poly(ethylene oxide-b-propylene oxide-b-ethylene oxide) block copolymer (BCP) as a nanostructured component. We establish a methodology to evaluate the effectiveness of a pre-shear protocol and evaluate several methods to identify a pre-shear procedure that resulted in repeatable transient creep results on a rheometer. We identified that large amplitude oscillatory shear was the most effective method for these materials, and the optimal magnitude of the shear was dependent on the composition of the epoxy resins. Through the consistent application of this approach, we were able to use transient creep testing to identify the phase boundaries in the epoxy/BCP resins when the BCP micelles undergo an order-order transition from spherical to hexagonal micelles through changes in the yield stress of the material. This work adds to the new growing body of literature demonstrating the importance of establishing rigorous pre-shear conditions to improve the accuracy of structured yield stress fluids. 

Federated learning at edge systems not only mitigates privacy concerns by keeping data localized but also leverages edge computing resources to enable real-time AI inference and decision-making. In a blockchain-based federated learning framework over edge clouds, edge servers as clients can contribute private data or computing resources to the overall training or mining task for secure model aggregation. To overcome the impractical assumption that edge servers will voluntarily join training or mining, it is crucial to design an incentive mechanism that motivates edge servers to achieve optimal training and mining outcomes. In this paper, we investigate the incentive mechanism design for a semi-asynchronous blockchain-based federated edge learning system. We model the resource pricing mechanism among edge servers and task publishers as a Stackelberg game and prove the existence and uniqueness of a Nash equilibrium in such a game. We then propose an iterative algorithm based on the Alternating Direction Method of Multipliers (ADMM) to achieve the optimal strategies for each participating edge server. Finally, our simulation results verify the convergence and efficiency of our proposed scheme. 

With the growing number of applications for thin polymer films (e.g., corrosion-resistant coatings, photovoltaics, and optoelectronics), there is an urgent need to develop or advance cost-effective, versatile, and high-throughput manufacturing processes to produce thin polymer films and coatings with controllable properties (e.g., morphology, composition). In this work, we present a simple, cost-effective, and scalable approach: the air-assisted electrospray method for thin film coating. We systematically investigate its capabilities for producing coatings with a wide range of surface morphologies, its compatibility with three-dimensional substrates, and the fundamental understanding of the process. Through systematic control of concentration, needle configuration, and polymer selection, we demonstrate the ability to produce coating morphologies with diverse structural characteristics and excellent reproducibility. Notably, the introduction of air assistance through a coaxial needle greatly enlarges the range of achievable morphologies, particularly at lower concentrations. We also found that the position of the airflow relative to the solution is critical for determining the polymer film properties. Furthermore, we demonstrate its broad application potential in the fabrication of binderless electrodes for sodium-ion batteries.